Stereo Image Point Cloud and Lidar Point Cloud Fusion for the 3d Street Mapping

Combining active and passive imaging sensors enables creating a more detailed 3D model of the real world. Then, these 3D data can be used for various applications, such as city mapping, indoor navigation, autonomous vehicles, etc. Typically, LiDAR and camera as imaging sensors are installed on these systems. Both of these sensors have advantages and drawbacks. Thus, LiDAR sensor directly provides relatively accurate 3D point cloud, but LiDAR point cloud barely contains the surface textures and details, such as traffic signs and alpha numeric information on facades. As opposed to LiDAR, deriving 3D point cloud from images require more computational resources, and in many cases, the accuracy and point density might be lower due to poor visual or light conditions. This paper investigates a workflow which utilizes factor graph SLAM, dense 3D reconstruction and ICP to efficiently generate the LiDAR and camera point clouds, and then, co-register in a navigation frame to provide a consistent and more detailed reconstruction of the environment. The workflow consists of three processing steps. First, we use factor graph SLAM, GPS/INS odometry and 6DOF scan matching to register the LiDAR point cloud. Then, the stereo images are processed by stereo-scan dense 3D reconstruction technique to generate dense point cloud. Finally, ICP method is used to co-register LiDAR and photogrammetric point clouds into one frame. The proposed method is tested with the KITTI dataset. The results show that data fusion of two point clouds can improve the quality of the 3D model.